Fix Readthedocs builds #577

peterzahemszky · 2021-11-23T16:43:06Z

Fix the Readthedocs builds.

Remove the traitlet-documenter dependency of Simphony Remote in favour of autodoc-traits. The GitHub repo of the former is archived, it only supports setuptools version 58.0.1 or older, and it makes the documentation build fail on Readthedocs. The licence of these two libraries is the same: BSD 3-Clause License (see https://github.com/jupyterhub/autodoc-traits/blob/main/LICENSE).
Remove the unnecessary code written for excluding imported members in the generated documentation. This is now a feature of the latest Sphinx version.
Fix a couple of minor issues with doc/source/conf.py and setup.py.

codecov-commenter · 2021-11-23T16:58:16Z

Codecov Report

Merging #577 (991d4ab) into master (6231b87) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #577   +/-   ##
=======================================
  Coverage   95.35%   95.35%           
=======================================
  Files          92       92           
  Lines        4305     4305           
  Branches      422      422           
=======================================
  Hits         4105     4105           
  Misses        140      140           
  Partials       60       60

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6231b87...991d4ab. Read the comment docs.

peterzahemszky · 2021-11-26T15:31:51Z

The proposed change seems to have resolved the original issue. The pip install -r doc-requirements.txt step is now successfully executing on Readthedocs.

However, the python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html command fails on Readthedocs with the following error:

Running Sphinx v1.4.4
making output directory...

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/simphony-remote/envs/11-23-fix-readthedocs-builds/lib/python3.7/site-packages/sphinx/cmdline.py", line 243, in main
    opts.warningiserror, opts.tags, opts.verbosity, opts.jobs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/simphony-remote/envs/11-23-fix-readthedocs-builds/lib/python3.7/site-packages/sphinx/application.py", line 137, in __init__
    confoverrides or {}, self.tags)
  File "/home/docs/checkouts/readthedocs.org/user_builds/simphony-remote/envs/11-23-fix-readthedocs-builds/lib/python3.7/site-packages/sphinx/config.py", line 291, in __init__
    execfile_(filename, config)
  File "/home/docs/checkouts/readthedocs.org/user_builds/simphony-remote/envs/11-23-fix-readthedocs-builds/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 130, in execfile_
    exec_(code, _globals)
  File "conf.py", line 28, in <module>
    from remoteappmanager.command_line_config import CommandLineConfig
  File "/home/docs/checkouts/readthedocs.org/user_builds/simphony-remote/envs/11-23-fix-readthedocs-builds/lib/python3.7/site-packages/remoteappmanager/command_line_config.py", line 1, in <module>
    import tornado.options
ModuleNotFoundError: No module named 'tornado'

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/simphony-remote/envs/11-23-fix-readthedocs-builds/lib/python3.7/site-packages/remoteappmanager/command_line_config.py", line 1, in <module>
    import tornado.options
ModuleNotFoundError: No module named 'tornado'
The full traceback has been saved in /tmp/sphinx-err-0jhzhlsz.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
mocking modules ['pamela', 'jupyterhub', 'jupyterhub.orm', 'jupyterhub.auth', 'jupyterhub.spawner', 'docker', 'docker.errors', 'tornadowebapi', 'tornadowebapi.authenticator', 'tornadowebapi.exceptions', 'tornadowebapi.http', 'tornadowebapi.registry', 'tornadowebapi.resource'] and types [('jupyterhub.orm', 'Proxy', (<class 'object'>,)), ('tornadowebapi.registry.orm', 'Registry', (<class 'object'>,))]

peterzahemszky · 2021-11-26T15:42:26Z

tornado is a runtime dependency of our remoteappmanager package, which is declared explicitly in the install_requires parameter of setuptools.setup(). However, it seems like the python setup.py install step didn't install it properly.

Possible issue on Stackoverflow: python setuptools install_requires is ignored when overriding cmdclass

Solutions:

replace self.run() with self.do_egg_install()
specify bdist_egg instead of install in cmdclass
move the npm build step out of setup.py
avoid using setup.py in favour of setup.cfg or pyproject.toml

peterzahemszky · 2021-11-26T16:11:37Z

The first solution works locally.

However, I now get yet another error when running Sphinx: RuntimeError: generator raised StopIteration.

Running Sphinx v1.4.4
mocking modules ['docker', 'docker.errors', 'tornadowebapi', 'tornadowebapi.authenticator', 'tornadowebapi.exceptions', 'tornadowebapi.http', 'tornadowebapi.registry', 'tornadowebapi.resource'] and types [('tornadowebapi.registry.orm', 'Registry', (<class 'object'>,))]
loading translations [en]... done
loading pickled environment... not yet created
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/ -> https://docs.python.org/3
[autosummary] generating autosummary for: administration.rst, api/remoteappmanager.application.rst, api/remoteappmanager.cli.remoteappdb.__main__.rst, api/remoteappmanager.cli.remoteapprest.__main__.rst, api/remoteappmanager.command_line_config.rst, api/remoteappmanager.db.csv_db.rst, api/remoteappmanager.db.interfaces.rst, api/remoteappmanager.db.orm.rst, api/remoteappmanager.docker.async_docker_client.rst, api/remoteappmanager.docker.container.rst, ..., design.rst, developer.rst, developer/ci_instructions.rst, developer/docker.rst, index.rst, troubleshoot/database.rst, troubleshoot/docker.rst, troubleshoot/general.rst, troubleshooting.rst, utilities.rst
WARNING: [autosummary] failed to import 'remoteappmanager.application': no module named remoteappmanager.application
WARNING: [autosummary] failed to import 'remoteappmanager.services.reverse_proxy': no module named remoteappmanager.services.reverse_proxy
WARNING: [autosummary] failed to import 'remoteappmanager.webapi.admin.application': no module named remoteappmanager.webapi.admin.application
WARNING: [autosummary] failed to import 'remoteappmanager.webapi.admin.container': no module named remoteappmanager.webapi.admin.container
WARNING: [autosummary] failed to import 'remoteappmanager.webapi.application': no module named remoteappmanager.webapi.application
WARNING: [autosummary] failed to import 'remoteappmanager.webapi.container': no module named remoteappmanager.webapi.container
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 40 source files that are out of date
updating environment: 40 added, 0 changed, 0 removed
reading sources... [ 20%] api/remoteappmanager.db.orm                                                                                                                                                           
Traceback (most recent call last):
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 656, in process_doc
    self.options, docstringlines)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 552, in emit
    results.append(callback(self, *args))
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/__init__.py", line 301, in _process_docstring
    obj, options)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/docstring.py", line 802, in __init__
    name, obj, options)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/docstring.py", line 159, in __init__
    self._parse()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/docstring.py", line 461, in _parse
    self._parsed_lines.extend(self._parse_attribute_docstring())
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/docstring.py", line 485, in _parse_attribute_docstring
    _type, _desc = self._consume_inline_attribute()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/docstring.py", line 239, in _consume_inline_attribute
    line = next(self._line_iter)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/iterators.py", line 65, in __next__
    return getattr(self, 'next')(n)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/napoleon/iterators.py", line 117, in next
    raise StopIteration
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/cmdline.py", line 244, in main
    app.build(opts.force_all, filenames)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 298, in build
    self.builder.build_update()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 251, in build_update
    'out of date' % len(to_build))
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 265, in build
    self.doctreedir, self.app))
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/environment.py", line 564, in update
    self._read_serial(docnames, app)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/environment.py", line 584, in _read_serial
    self.read_doc(docname, app)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/environment.py", line 737, in read_doc
    pub.publish()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/core.py", line 218, in publish
    self.settings)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/io.py", line 49, in read
    self.parse()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/__init__.py", line 183, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 171, in run
    input_source=document['source'])
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/statemachine.py", line 240, in run
    context, state, transitions)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/statemachine.py", line 240, in run
    context, state, transitions)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2097, in directive
    directive_class, match, type_name, option_presets)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 1613, in run
    documenter.generate(more_content=self.content)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 963, in generate
    self.document_members(all_members)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 1338, in document_members
    ModuleLevelDocumenter.document_members(self, all_members)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 885, in document_members
    check_module=members_check_module and not isattr)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 960, in generate
    self.add_content(more_content)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 1505, in add_content
    ClassLevelDocumenter.add_content(self, more_content, no_docstring)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 694, in add_content
    for i, line in enumerate(self.process_doc(docstrings)):
RuntimeError: generator raised StopIteration

Exception occurred:
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc.py", line 694, in add_content
    for i, line in enumerate(self.process_doc(docstrings)):
RuntimeError: generator raised StopIteration
The full traceback has been saved in /tmp/sphinx-err-bi845p0b.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

peterzahemszky · 2021-11-26T16:31:33Z

However, I now get yet another error when running Sphinx: RuntimeError: generator raised StopIteration.

This seems to be this issue: sphinx-doc/sphinx#5076. It's fixed in Sphinx version 1.7: sphinx-doc/sphinx@b553c23. I can also confirm that it's not reproducible on Python 3.6.

peterzahemszky · 2021-11-26T16:43:06Z

Removing the Sphinx version constraint from doc-requirements.txt (and using Sphinx 1.8.6) resolves the above issue.

However, running the Sphinx command gives yet another error: ImportError: cannot import name 'Directive' from 'sphinx.util.compat'

Running Sphinx v4.3.0
mocking modules ['docker', 'docker.errors', 'tornadowebapi', 'tornadowebapi.authenticator', 'tornadowebapi.exceptions', 'tornadowebapi.http', 'tornadowebapi.registry', 'tornadowebapi.resource'] and types [('tornadowebapi.registry.orm', 'Registry', (<class 'object'>,))]
loading translations [en]... done

Traceback (most recent call last):
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/registry.py", line 429, in load_extension
    mod = import_module(extname)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/__init__.py", line 70, in <module>
    from sphinx.util.compat import Directive
ImportError: cannot import name 'Directive' from 'sphinx.util.compat' (/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/util/compat.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/cmd/build.py", line 279, in build_main
    args.tags, args.verbosity, args.jobs, args.keep_going)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 237, in __init__
    self.setup_extension(extension)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 394, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/registry.py", line 433, in load_extension
    err) from err
sphinx.errors.ExtensionError: Could not import extension _extensions (exception: cannot import name 'Directive' from 'sphinx.util.compat' (/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/util/compat.py))

Extension error:
Could not import extension _extensions (exception: cannot import name 'Directive' from 'sphinx.util.compat' (/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/util/compat.py))

peterzahemszky · 2021-11-26T17:33:07Z

According to https://www.sphinx-doc.org/en/master/extdev/deprecated.html, sphinx.util.compat.Directive is deprecated in favour of docutils.parsers.rst.Directive.

Replacing the relevant import in doc/source/_extensions/init.py makes the above mentioned error go away.

However, there's another issue that comes up afterwards:

  File "/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/__init__.py", line 558, in process_generate_options
    ext = app.config.source_suffix[0]
KeyError: 0

Running Sphinx v1.8.6
mocking modules ['docker', 'docker.errors', 'tornadowebapi', 'tornadowebapi.authenticator', 'tornadowebapi.exceptions', 'tornadowebapi.http', 'tornadowebapi.registry', 'tornadowebapi.resource'] and types [('tornadowebapi.registry.orm', 'Registry', (<class 'object'>,))]
loading translations [en]... done
loading pickled environment... failed: Can't get attribute 'WarningStream' on <module 'sphinx.util.nodes' from '/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/util/nodes.py'>
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:43: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(ModuleDocumenter)
/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py:1540: RemovedInSphinx20Warning: AutoDirective._registry has been deprecated. Please use app.add_autodocumenter() instead.
  AutoDirective._registry[cls.objtype] = cls
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:44: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(ClassDocumenter)
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:45: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(ExceptionDocumenter)
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:46: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(DataDocumenter)
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:47: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(FunctionDocumenter)
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:48: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(MethodDocumenter)
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:49: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(AttributeDocumenter)
/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/generate.py:50: RemovedInSphinx20Warning: sphinx.ext.autodoc.add_documenter() has been deprecated. Please use app.add_autodocumenter() instead.
  add_documenter(InstanceAttributeDocumenter)

Traceback (most recent call last):
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/cmd/build.py", line 303, in build_main
    args.tags, args.verbosity, args.jobs, args.keep_going)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 263, in __init__
    self._init_builder()
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 325, in _init_builder
    self.emit('builder-inited')
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/application.py", line 510, in emit
    return self.events.emit(event, self, *args)
  File "/home/pzahemszky/Repositories/simphony-remote/py37env/lib/python3.7/site-packages/sphinx/events.py", line 80, in emit
    results.append(callback(*args))
  File "/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/__init__.py", line 558, in process_generate_options
    ext = app.config.source_suffix[0]
KeyError: 0

Exception occurred:
  File "/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/__init__.py", line 558, in process_generate_options
    ext = app.config.source_suffix[0]
KeyError: 0
The full traceback has been saved in /tmp/sphinx-err-xi66isbc.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

peterzahemszky · 2021-11-26T18:06:47Z

The above issue seems to be a problem with our doc/source/_extensions/init.py file, in particular its process_generate_options() function. Writing list(app.config.source_suffix)[0] (i.e. wrapping the source_suffix mapping in a list) resolves the above error.

peterzahemszky · 2021-11-26T18:25:49Z

Readthedocs build successfully completed: https://readthedocs.org/projects/simphony-remote/builds/15383366/ (with lots of warnings)

peterzahemszky · 2021-11-26T18:33:05Z

The CI failure at https://github.com/simphony/simphony-remote/runs/4336809201 is an issue with doc/source/conf.py.

It can be fixed by changing autodoc_member_order from 'source' to 'bysource' to cater for the new Sphinx version used.

peterzahemszky · 2021-11-26T18:43:46Z

sphinx-build -W doc/source doc/build/sphinx failed on the Ubuntu CI (https://github.com/simphony/simphony-remote/runs/4336888742):

Running Sphinx v4.3.0
mocking modules [] and types []
making output directory... done
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv

Extension error (_extensions):
Handler <function process_generate_options at 0x7f735a5d6620> for event 'builder-inited' threw an exception (exception: cannot import name 'add_documenter')
make: *** [docs] Error 2
Makefile:218: recipe for target 'docs' failed

peterzahemszky · 2021-11-26T19:30:28Z

Handler <function process_generate_options at 0x7f735a5d6620> for event 'builder-inited' 
threw an exception (exception: cannot import name 'add_documenter')

From the changelog of Sphinx release 1.7.0:

sphinx.ext.autodoc.add_documenter() and AutoDirective._register is now deprecated. Please use app.add_autodocumenter() instead.

peterzahemszky · 2021-11-26T19:45:00Z

CI now gives the following error:

directive 'automodule' is already registered, it will be overridden

peterzahemszky · 2021-12-03T17:04:28Z

directive 'automodule' is already registered, it will be overridden

This can be resolved by removing some documenter classes from doc/source/_extensions/__init__.py that are duplicate:

simphony-remote/doc/source/_extensions/__init__.py

Lines 561 to 567 in 49e06a1

    
           app.add_autodocumenter(ModuleDocumenter) 
        
           app.add_autodocumenter(ClassDocumenter) 
        
           app.add_autodocumenter(ExceptionDocumenter) 
        
           app.add_autodocumenter(DataDocumenter) 
        
           app.add_autodocumenter(FunctionDocumenter) 
        
           app.add_autodocumenter(MethodDocumenter) 
        
           app.add_autodocumenter(AttributeDocumenter)

This seems to resolve the above issue.

peterzahemszky · 2021-12-03T17:04:40Z

There's a new error message saying 'Sphinx' object has no attribute 'warn':

$ make docs
sphinx-build -W doc/source doc/build/sphinx
Running Sphinx v4.3.0
mocking modules [] and types []
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv

Extension error (_extensions):
Handler <function process_generate_options at 0x7fb442c6ac80> for event 'builder-inited' threw an exception (exception: 'Sphinx' object has no attribute 'warn')
Makefile:218: recipe for target 'docs' failed
make: *** [docs] Error 2

peterzahemszky · 2021-12-03T17:09:44Z

There's a new error message saying 'Sphinx' object has no attribute 'warn'

This can be worked around by removing the use of the deprecated methods info() and warn().

peterzahemszky · 2021-12-03T17:24:02Z

make docs is now giving an error that says cannot import name 'AutoDirective':

$ make docs
sphinx-build -W doc/source doc/build/sphinx
Running Sphinx v4.3.0
mocking modules [] and types []
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv
[autosummary] generating autosummary for: administration.rst, api/remoteappmanager.application.rst, api/remoteappmanager.cli.remoteappdb.__main__.rst, api/remoteappmanager.cli.remoteapprest.__main__.rst, api/remoteappmanager.command_line_config.rst, api/remoteappmanager.db.csv_db.rst, api/remoteappmanager.db.interfaces.rst, api/remoteappmanager.db.orm.rst, api/remoteappmanager.docker.async_docker_client.rst, api/remoteappmanager.docker.container.rst, ..., design.rst, developer.rst, developer/ci_instructions.rst, developer/docker.rst, index.rst, troubleshoot/database.rst, troubleshoot/docker.rst, troubleshoot/general.rst, troubleshooting.rst, utilities.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 40 source files that are out of date
updating environment: [new config] 40 added, 0 changed, 0 removed
reading sources... [  5%] api/remoteappmanager.application                                                             
Exception occurred:
  File "/home/pzahemszky/Repositories/simphony-remote/doc/source/_extensions/__init__.py", line 153, in get_documenter
    from sphinx.ext.autodoc import AutoDirective, DataDocumenter, \
ImportError: cannot import name 'AutoDirective'
The full traceback has been saved in /tmp/sphinx-err-mvk03zxm.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
Makefile:218: recipe for target 'docs' failed
make: *** [docs] Error 2

peterzahemszky · 2021-12-03T17:24:18Z

I'll stop working on this PR for now

peterzahemszky · 2021-12-15T18:44:24Z

I'm planning to revive this PR again

peterzahemszky · 2021-12-15T19:17:07Z

make docs is now giving an error that says cannot import name 'AutoDirective'

From https://www.sphinx-doc.org/en/master/extdev/deprecated.html it looks like sphinx.ext.autodoc.AutoDirective._registry is deprecated. We'd need to find an equivalent of AutoDirective._registry.values() here:

simphony-remote/doc/source/_extensions/__init__.py

Lines 170 to 176 in 6231b87

    
           classes = [cls for cls in AutoDirective._registry.values() 
        
                      if cls.can_document_member(obj, '', False, parent_doc)] 
        
           if classes: 
        
               classes.sort(key=lambda cls: cls.priority) 
        
               return classes[-1] 
        
           else: 
        
               return DataDocumenter

peterzahemszky · 2021-12-15T22:04:26Z

Having found

simphony-remote/doc/source/_extensions/generate.py

Lines 156 to 158 in 6231b87

    
           # Patched get_members according to 
        
           # http://stackoverflow.com/questions/25405110/sphinx-autosummary-with-toctree-also-lists-imported-members/25460763#25460763 
        
           def get_members(obj, typ, include_public=[], imported=False):

and the referenced Stack Overflow question, I realised that probably the doc/source/_extensions/__init__.py and doc/source/_extensions/generate.py files are purely for avoiding documentation generation for imported members.

According to the Sphinx changelog for Release 1.6.1, this is now the default behaviour for the latest version. Consequently, we can get rid of these two modules safely

peterzahemszky · 2021-12-15T23:45:19Z

The documentation build succeeds on both CIs, as well as on Readthedocs. This PR is now ready for review

peterzahemszky · 2021-12-16T12:55:14Z

Given that this set of changes is fairly simple and contained, I propose merging this PR now (and addressing potential issues in follow-up PRs).

@flongford

peterzahemszky added 2 commits November 23, 2021 16:27

Remove the traitlet-documenter dependency, add autodoc-traits

786cde6

Update configuration of the Sphinx documentation

a207e7b

peterzahemszky added 3 commits November 26, 2021 17:57

Fix missing runtime dependencies

7f62b58

Remove Sphinx version constraint to resolve issue

2398f3e

Resolve Sphinx and docutils issues

d4d4d2a

Fix flake8 issue

e66a1fa

Fix autodoc_member_order in Sphinx config

99cd76f

Use app.add_autodocumenter to resolve Sphinx failure

49e06a1

Remove duplicate documenter classes

4fd167b

Remove deprecated methods

9c64a75

Clean up imports

193eda3

Remove redundant code

991d4ab

peterzahemszky changed the title ~~[WIP] Fix Readthedocs builds~~ Fix Readthedocs builds Dec 15, 2021

peterzahemszky merged commit 6f4338b into master Dec 16, 2021

peterzahemszky deleted the 11-23-fix-readthedocs-builds branch December 16, 2021 12:55

peterzahemszky mentioned this pull request Dec 16, 2021

Introduce Readthedocs config file #586

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Readthedocs builds #577

Fix Readthedocs builds #577

peterzahemszky commented Nov 23, 2021 •

edited

Loading

codecov-commenter commented Nov 23, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 3, 2021 •

edited

Loading

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 16, 2021

Fix Readthedocs builds #577

Fix Readthedocs builds #577

Conversation

peterzahemszky commented Nov 23, 2021 • edited Loading

codecov-commenter commented Nov 23, 2021 • edited Loading

Codecov Report

peterzahemszky commented Nov 26, 2021 • edited Loading

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021 • edited Loading

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021 • edited Loading

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Nov 26, 2021

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 3, 2021 • edited Loading

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 3, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 15, 2021

peterzahemszky commented Dec 16, 2021

peterzahemszky commented Nov 23, 2021 •

edited

Loading

codecov-commenter commented Nov 23, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021 •

edited

Loading

peterzahemszky commented Nov 26, 2021 •

edited

Loading

peterzahemszky commented Dec 3, 2021 •

edited

Loading